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ABSTRACT 


I The  paper  describes  a state-of-the-art  com- 
i puter-based  project  risk  analysis  technique 
which  has  been  In  widespread  use  since  1970. 

The  technique  has  been  used  to  assist  In 
management  of  project  risks  on  over  a hundred 
projects  worldwide,  with  a total  value  of  over 
$55  billion.  PROMAP  V*^resembles  conventional 
deterministic  project  management  tools  only  In 
that  It  uses  the  conventional  critical  path 
network  as  the  framework  for  a project  risk 
model.  The  model  Is  then  analyzed  to  deter- 
mine the  Interrelated  schedule  and  cost  risks 
resulting  from  time,  cost,  and  technical  per- 
formance uncertainties.  At  the  same  time,  the 
model  serves  In  the  usual  way  the  routine 
functions  of  project  estimating,  planning, 
scheduling,  resourcing,  costing,  and  control. 
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BACKGROUND  AND  INTRODUCTION 

In  1963  and  1964,  as  a consequence  of  a number 
of  significant  Defense  program  overruns,  the 
RAND  Corporation  Investigated  the  analytical 
assumptions  of  the  PERT  project  management 
tool  which  had  been  utilized  on  an  Increasing 
number  of  D00  programs  over  the  previous  five- 
year  period.  The  results  of  the  research  !■) 
revealed  that  a major  shortcoming  of  the  criti- 
cal path  technique  Is  that  because  It  Is  deter- 
ministic It  does  not  adequately  account  for  the 
Impact  of  uncertainties  on  project  time  and 
cost  performance.  As  a consequence,  PERT- 
based  schedules  and  time-related  estimates  of 
costs  are  Inherently  optimistic,  and  project 
overruns  are  being  Inadvertently  built  Into  the 
project  plan  from  the  very  start. 

Almost  twenty  years  later,  the  Defense  systems 
acquisition  community  now  finds  Itself  In  the 
very  peculiar  circumstance  of  continuing  to 
support  widespread  use.  of  deterministic  project 
management  tools  on  one  hand,  while  on  the 
other,  attempting  to  cope  with  the  problems  of 
Increasingly  complex  acquisition  programs  by 
sponsoring  development  of  a proliferation  of 
special-purpose  analytical  tools  designed 
specifically  to  deal  with  project  risks  and 
uncertainties. 

T7)  See  References  5,  9,  10  and  11. 

♦PROMAP  V Is  a service  mark  of  Log/An,  Inc.U.S.A. 
Copyright  (C)  1983  Log/An,  Inc. 


Project  management  has  been  employed  on  DOD 
programs  for  some  time,  but  until  recently,  it 
has  been  largely  non-rigorous.  The  original 
PERT  was  Intended  to  rigorously  deal  with  un- 
certainties affecting  project  time  and  cost 
performance,  and  at  that  time,  It  was  des- 
cribed as  a "stochastic"  technique  because  It 
accepts  range  estimate  Inputs  which  account  for 
uncertainty.  But,  the  range  values  are  re- 
duced to  a single  "expected"  value  and  the 
subsequent  critical  path  analysis  Is  determi- 
nistic. Hence,  the  benefits  of  probabilistic 
analysis  are  not  realized. 

At  UCLA,  in  1966,  we  started  development  of  a 
probabilistic  network  analysis  package  de- 
signed specifically  for  projects  where  uncer- 
tainties are  significant.  By  1968,  we  were 
applying  the  first  operational  versions  of 
PROMAP  (Project  Risk  Management  and  Planning) 
on  Navy  ship  overhaul  projects.  Early  appli- 
cation results  were  reported  In  the  Navy 
Management  Review,  April/May  1969  Issue. 

Navy  applications  continued  on  ship  acquisition, 
modernization,  and  repair  projects  and  In  1975, 
at  the  Fourth  Annual  DOD  Procurement  Research 
Symposium  at  Colorado  Springs,  I described  the 
PROMAP  approach  and  compared  It  with  conven- 
tional deterministic  techniques  with  the  aid  of 
explicit  results. 

The  following  year,  the  PROMAP  technique  was 
Included  In  the  Naval  Sea  Systems  Command  pro- 
ject management  handbook,  Reef  Points,  Later 
In  1976,  I presented  a paper  at  the  Fifth 
Annual  DOD  Procurement  Research  Symposium  at 
Monterey,  covering  application  of  PROMAP  to 
government  contract  claims  analysis. 

Further  applications  on  DOD  and  non-Defense 
projects  In  aerospace,  energy,  transportation, 
and  construction  fields  led  to  continuing  Im- 
provement in  the  PROMAP  approach,  and  In  1981 
as  Panel  Chairman  at  the  Air  Force  Risk  and 
Uncertainty  Workshop  at  Colorado  Springs,  I 
discussed  the  advanced  features  of  the  latest 
version,  PROMAP  V. 

Over  the  15  years  since  It  became  operational, 
the  PROMAP  V technique  and  Its  predecessors 
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have  been  successfully  applied  to  over  a 
hundred  large,  complex  Defense  and  non-Defense 
projects,  with  a total  value  In  excess  of  $55 
billion. 


THE  PROMAP  V*  APPROACH 

The  foundation  of  the  PROMAP  V*  approach  Is  the 
Project  Risk  Model  which  Is  In  effect,  the 
conventional  critical  path  activity  network, 
modified  to  Include  logic  and  data  accounting 
for  uncertainties  In  (see  Figure  1): 


Internal  Factors 

. Planning  (Including 
contingency  planning) 

. Technical  performance 

. Time  performance 

. Resource  performance 

. Cost  performance 

. System  support 
readiness 


External  Factors 
. Economic  factors 
. Funding 

. Environmental 
factors 

. External 
deliverables 


By  accounting  for  the  various  types  of  uncer- 
tainty In  a single  model,  the  computer  analysis 
accomplishes  the  Intricate  correlations  among 
the  different  uncertainty  factors  and  schedules 
and  costs,  which  Is  so  necessary  for  a re- 
liable assessment  of  project  risks.  Data  In- 
puts are  accepted  from  a variety  of  reliable 
data  sources  and  estimating  approaches,  In- 
cluding empirical  data  and  parametric, 
engineering,  analogy,  factor,  and  subjective 
estimating  techniques. 

In  conducting  a project  risk  analysis  with 
PROMAP  V*,  the  project  model  Is  "run"  (simu- 
lated) In  the  computer  as  many  as  several 
hundred  times;  each  run  representing  a complete 
project  realization  from  start  to  end,  with 
activity  paths,  activity  durations,  resource 
requirements,  and  costs  sampled  from  distri- 
butions contained  In  the  Input  data. 

The  results  Include  project  schedules  and 
schedule  risks,  costs  and  cost  risks,  and  re- 
source requirements,  together  with  data  on 
critical  activities,  diagnostics,  graphics, 
and  other  management  information. 


PROMAP  V PROJECT  RISK  ANALYSIS 
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Figure  1.  PROMAP  V*  PROJECT  RISK  ANALYSIS 
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A.  Planning  and  Contingency  Planning 

’ The  conventional  critical  path  network  (pro- 
! ject  plan).  Identifies  all  project  activities 
from  start  to  completion.  Activities  are 
arranged  In  proper  sequence  of  performance,  de- 
picting their  Interrelations  and  Interde- 
pendencies. Conventional,  deterministic 
techniques  are  limited  to  representing  each 
activity  at  a 100  per  cent  likelihood  of 
occurrence. 


A key  feature  of  the  PROMAP  V*  technique  Is  Its 
ability  to  account  for  uncertainty  In  the  pro- 
ject plan.  For  example,  a set  of  operational 
specifications  may  or  may  not  be  returned  to 
the  preparer  for  revisions.  Or  certain  soft- 
ware design  features  may  or  may  not  be  re- 
jected by  the  Project  Manager  and  returned  for 
modification;  a "backup"  plan  may  be  undertaken 
to  substitute  a less  advanced  state-of-the-art 
system  feature  should  the  primary  design  effort 
prove  to  be  unsuccessful;  or  weather  may  delay 
an  Important  field  test. 

If  the  program  plan,  schedule,  costs,  and  re- 
source requirements  are  to  be  realistic,  such 
uncertain  actions  must  be  accounted  for  In 
terms  of  their  likelihood  of  occurring. 


As  an  example.  Figure  2 Illustrates  a project 
plan  Incorporating  a contingency  plan  con- 
sisting of  Activities  5 and  8,  representing  a 
"back-up"  in  the  likelihood  that  upon  com- 
pletion of  Activity  2,  the  primary  plan  to  de- 
velop an  advanced  state-of-the-art  system 
feature  (Activities  4,  7)  will  be  assessed  to 
be  too  risky.  At  the  start,  it  is  assessed 
that  the  primary  plan  has  an  80%  probability 
of  technical  success.  Accordingly,  the  "back- 
up" plan  is  assigned  a 20%  probability  of  being 
implemented.  This  is  referred  to  as  "Con- 
tingency Planning." 

As  another  example,  suppose  Activity  6 is  a key 
test  of  a critical  subassembly.  Based  on  past 
experience,  it  Is  estimated  that  there  Is  a 90% 
chance  that  the  test  will  be  successful 
(Activity  9)  and  a 10%  likelihood  that  it  will 
fall  and  the  subassembly  will  have  to  undergo 
some  modifications  and  retest  (Activity  10). 
This  situation  Is  depicted  by  showing  Activity 
9 as  having  a 90%  probability  of  occurring, 
and  Activity  10  a 10%  probability  of  occurring. 


CONTINGENCY  PLANNING 


Figure  2.  Contingency  Planning 
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8.  Technical  Risk  Management 

Uncertainties  and  risks  Inherent  In  technical 
or  software  development  can  have  major  Impact 
on  the  likelihood  of  attaining  project  object- 
ives. 

In  the  typical  case,  the  technical  risk  ele- 
ments are  Identified  and  the  risks  assessed. 
Conventionally,  for  technical  risk  elements 
which  are  critical  to  project  success,  manage- 
ment attention  Is  directed  at  minimizing  the 
Impact  on  project  performance,  ullth  PROMAP  V*, 
"contingency"  plans  designed  to  mitigate  the 
risks  are  developed  and  Included  as  part  of  the 
overall  project  model.  Typically,  contingency 
planning  may  Include  measures  such  as  early 
starts,  allocation  of  additional  resources, 
redundancy,  and  substitution  of  proven  state- 
of-the-art  technology. 

During  the  course  of  the  project,  as  the  tech- 
nical development  proceeds,  periodic  reviews 
are  made  to  obtain  a current  risk  assessment 
of  the  technical  risk  elements. 

It  Is  normally  expected  that  the  risk  level  for 
an  Individual  development  Item  will  decrease 
as  the  work  progresses.  However,  should  the 
updated  risk  assessment  Indicate  that  the 
technical  risk  level  has  not  adequately  de- 
creased since  the  previous  assessment,  the 
appropriate  measures  are  taken  to  accelerate 
Implementation  of  related  contingency  plans; 
the  objective  Is  to  provide  assurance  that 
project  objectives  are  attained  despite  the 
continued  existence  of  technical  risks. 

C.  Schedule  Risk  Analysis 

Schedule  risk  analysis  results  Include  the 
range  of  times  covering  the  span  between  the 
earliest  and  latest  possible  dates  for  project 
completion  and  Individual  milestones,  with 
accompanying  detailed  activity  schedules  and 
schedule  risks.  Figure  3 Illustrates  the 
range  of  project  completion  times  for  an  ex- 
ample project.  The  results  show  that  the  pro- 
ject might  be  completed  In  from  240  to  330 
workdays.  The  cumulative  plot  presents  the 
probabilities  of  project  duration  between  the 
two  extremes.  For  example,  there  Is  a 90* 
probability  that  the  project  will  be  completed 
In  300  workdays  or  less,  a 60*  probability  In 
280  workdays  or  less,  and  so  forth.  The  re- 
sults show  that  there  Is  an  85  per  cent  chance 
of  overrunning  the  schedule  target  of  260  work- 
days. 

Activity  Criticality 

One  of  the  misleading  aspects  of  conventional 
deterministic  methods  Is  the  assumption  that 
there  Is  a single  "critical"  (longest  time) 
path  which  determines  the  duration  of  the  pro- 
ject. The  designated  critical  path  then  be- 
comes a focal  point  for  project  management. 


T'me  Summary  Graph 

PftOXCT  MOOtl  EXAMPLE  OVf HALL  COMPLETION  TIME 


Figure  3.  Time  Summary  Graph 


However,  when  uncertainty  factors  are  accounted 
for,  there  can  be  a number  of  different  net- 
work paths  which  have  significant  probability 
of  becoming  critical  during  the  course  of  the 
project.  In  fact,  a probabilistic  analysis 
will  usually  demonstrate  that  the  "longest 
time"  path  of  the  deterministic  technique  has 
significantly  less  than  a 100*  probability  of 
becoming  critical. 

For  the  sake  of  precise  project  monitoring  and 
control,  risk  analysis  focuses  on  activity 
"criticality";  the  higher  the  criticality 
value  (In  per  cent)  - which  Is  an  output  of  the 
analysis  - the  more  sensitive  the  activity, 
with  regard  to  overall  project  schedule  per- 
formance. 

Accordingly, \the  activities  requiring  close 
managerial  attention  are  those  with  the  most 
criticality.  On  some  projects,  as  many  as  35 
to  50  per  cent  of  the  project  activities  have 
a significant  level  of  criticality. 

D.  Resource  Risk  Analysis 

Schedules  and  budgets  are  not  realistic  unless 
the  resources  required  to  accomplish  the  indi- 
vidual project  activities  are  available  when 
needed.  Resources  may  Include  personnel , 
materials,  documentation,  equipment,  facilities, 
funds,  or  suitable  environmental  conditions. 
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Most  projects  suffer  from  some  scarcity  of  re- 
sources--the  net  effect  can  be  to  significantly 
delay  the  project  completion  and  add  to  the 
project  cost. 

PROMAP  V*  provides  the  project  manager  with 
resource  requirements,  taking  Into  account  the 
variable  start  and  finish  dates  of  the  activi- 
ties. 

Figures  4 and  5 show  the  difference  between 
deterministic  and  probabilistic  resource 
analyses.  The  deterministic  results  In  Figure 
4 show  a one-day  peak  requirement  of  nine 
General  Maintenance  Men;  Figure  5 shows  a 
probabilistic  requirement  of  none  for  as  long 
as  twenty  days.  The  difference  Is  due  to  the 
cumulative  Impact  of  uncertainties  In  the 
probabilistic  case. 


Resource  Requirements  Graph 

DTTEflMNSTC  ANALYSIS 

Retotroc  Cod*  MAWTMAN 
Description:  GENERAL  MAINTENANCE  MEN 


MAM  PROJECT  TITLE  RUN  ♦ DATE 

Resource  History  Graph 
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Resource  Code  MAMTMAN 
Description  GENERAL  MAMTE NANCE  MEN 
Number  of  Resource  Umti:  14 


Figure  5.  Probabilistic  Resource  Requirements 


E.  Cost  Risk  Analysis 

Conventionally,  cost  estimates  are  determinis- 
tic; that  Is,  costs  for  the  Individual  line 
Items  of  a project  budget,  or  for  Individual 
project  activities  of  a network,  are  expressed 
as  single  values  representing  perhaps  the 
"best"  1./  estimate.  However,  where  there  Is 
uncertainty,  the  use  of  range  estimates  allows 
a "cost  risk  analysis"  which  combines  the  un- 
certainties for  the  different  cost  elements 
and  determines  the  overall  range  of  project 
costs  between  minimum  and  maximum  and  as  well 
as  the  risks  of  overrunning  project  cost 
targets. 


Figure  4.  Deterministic  Resource  Requirements 


T77  The  practice  varies  considerably:  "best", 

'"’most  likely",  "average",  "normal",  or  no 
special  designation  at  all  may  be  given  to  the 
estimate.  On  many  projects,  there  Is  no  spe- 
cific standard  discipline  applied  to  the 
estimating  process. 
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Typical  results  of  a cost  risk  analysis  are 
shown  In  Figure  6.  The  range  of  total  project 
costs  Is  given  together  with  the  probabilities 
of  different  cost  outcomes  between  the  two 
extreme  values.  For  example.  It  Is  shown  that 
there  Is  a 60X  probability  of  expenditures 
reaching  the  amount  of  $60  million  (hence,  a 
40k  probability  of  exceeding  that  amount,  and 
by  as  mvch  as  $12.5  million). 


Cost  Summary  Graph 

TOTAL  PROJECT  COST  (E.O.J.  DOLLARS) 


total  cost 

(ULLIONS  OF  DOLLARS) 


Figure  6.  Cost  Summary  Graph 
Total  Project  Cost  (E.O.J.  Dolla.s) 


F.  Cost/Schedule  Analysis 

Figure  7 shows  a PROMAP  V*  cost/schedule  graph 
for  a typical  project.  The  projection  to  com- 
pletion Incorporates  the  uncertainties  regard- 
ing future  events  and  produces  a "projected 
outcome  area"  which  Includes  all  completion 
possibilities  between  the  extremes  In  time  and 
cost  performance.  A specific  cost/schedule 
target  value  may  be  selected  as  representing 
any  specified  level  of  risk  acceptable  to 
management.  In  Figure  7,  the  target  value 
shown  Is  the  "expected"  cost/schedule  value, 
which  has  an  average  likelihood  of  being 
realized. 


Cost  / Schedule  Risk  Analysis 

(AT  START) 


Figure  7.  Cost/Schedule  Projected  Performance 

Figure  8 shows  the  cost/schedule  performance 
at  a later  stage  of  the  project.  Typically, 
the  size  of  the  projected  outcome  area  de- 
creases as  certainty  replaces  uncertainty  as 
the  work  proceeds.  A major  task  In  project 
risk  management  is  to  assure  that  the  ex- 
pected value  of  the  projected  outcome  area 
does  not  materially  deviate  from  the  target 
value,  as  Is  shown  In  Flgurr  8. 


Cost  / Schedule  Risk  Analysis 

■ uNOERWAV  ■ STAGE  ( 


Figure  8.  Cost/Schedule  Risk  Analysis 
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PROMAP  V*  provides  the  diagnostics  to  allow 
the  project  manager  to  make  the  specific  ad- 
justments to  bring  the  projected  outcome  area 
Into  an  acceptable  risk  range  (as  shown  In 
Figure  9). 


Cost  / Schedule  Risk  Analysis  - Adjusted 

■ uNOORWAV  ■ STACIE  I 


Figure  9.  Cost/Schedule  Risk  Analysis-Adjusted 


LESSONS  LEARNED 

1.  Budgets  and  schedules  should  be  based  on 
the  results  of  risk  analyses  with  "expected" 
values  chosen  for  the  project  targets  and  the 
use  of  a contingency  allowance  or  management 
reserve  should  be  avoided.  In  practice,  the 
contingency  allowance  Is  set-aside  to  cover 
the  Impact  of  uncertainties  on  project  cost 
performance.  However,  project  uncertainties 
generate  "minus"  as  well  as  "plus"  possibili- 
ties; events  may  turn  out  better  than  expected. 
Because  the  conventional  contingency  allowance 
covers  just  the  "pluses",  there  Is  no  planning 
to  take  advantage  of  the  "minus"  Instances 
when  they  occur.  Adopting  "expected"  values 
and  contingency  planning  will  reduce  the  risks 
and  with  effective  project  risk  management, 
overrun  possibilities  will  be  minimized. 

2.  A baseline  risk  analysis  should  be  con- 
ducted early  In  the  program  and  updated  as 
appropriate  during  the  pre-award  period,  assur- 
ing the  availability  of  a current  risk  base- 
line at  the  time  of  source  evaluation. 


3.  The  system  acquisition  RFP  should  specify 
that  the  bidder  support  Its  presentation  with 
the  following  data: 

a.  Identification  of  the  cost,  schedule, 
and  technical  risk  elements. 

b.  Description  of  contingency  plans  de- 
signed to  reduce  risks  on  critical  technical 
elements  to  manageable  levels. 

c.  Results  of  a risk  analysis  covering 
cost,  schedule,  and  technical  risks. 

d.  Explanation  of  the  risk  assessment 
justification  supporting  the  bid  price. 

e.  Description  of  the  bidder's  risk 
management  plan  covering  cost,  scheduling,  and 
technical  risks,  together  with  the  details  of 
procedures  and  system  for  implementing  the 
risk  management  plan. 

4.  As  part  of  the  bid  evaluation,  the 
government  should  compare  the  bidder's  risk 
analysis  results  with  the  government's  base- 
line values.  Any  significant  difference 
should  be  analyzed. 

5.  Once  the  contract  Is  awarded,  the  govern- 
ment should  continually  monitor  project  risks 
and  risk  trends,  the  latter  providing  a very 
sensitive  Indicator  of  problems  ahead.  The 
periodic  reviews  of  contractor  performance 
should  encompass  cost,  schedule,  and  technical 
risk  considerations  In  addition  to  the  standard 
requirements  of  DOD  I 7000.2. 
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